Statistical Mechanics of Broadcast Channels Using 
Low Density Parity Check Codes 



Kazutaka Nakamuraf , Robert Morelos-Zaragoza$ , David 
Saad§ and Yoshiyuki Kabashimaf 

f Department of Computational Intelligence and Systems Science, Tokyo Institute of 
Technology, Yokohama 226-8502, Japan 

X Advanced Telecommunication Laboratory, SONY Computer Science Laboratories, 
Inc., Shinagawa-ku, Tokyo 141-0022, Japan 

§ The Neural Computing Research Group, School of Engineering and Applied 
Science, Aston University, Birmingham B4 7ET, UK 

Abstract. We investigate the use of Gallager's low-density parity-check (LDPC) 
codes in a broadcast channel, one of the fundamental models in network information 
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communication schemes and is known to provide better performance than simple 
timesharing methods when algebraic codes are used. The statistical physics based 
analysis shows that the practical performance of the suggested method, achieved 
by employing the belief propagation algorithm, is superior to that of LDPC based 
timesharing codes while the best performance, when received transmissions are 
optimally decoded, is bounded by the timesharing limit. 
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1. Introduction 

Progress in digital communication technologies has dramatically increased the 
information flow in both wired and wireless channels. This makes the role of generic 
coding techniques, such as error- correcting codes and data compression, more important. 
As most existing codes are constructed for simple point-to-point communication, they 
do not necessarily provide optimal performance in multi-terminal communication such 
as the inter-net, mobile phones and satellite communication. Therefore, designing 
improved codes that utilize characteristic properties of these media is a promising 
direction for enhancing the performance of multi-terminal communication. 

The broadcast channel is a standard multi-terminal communication channel 
composed of a single sender and multiple receivers, and is characteristic of TV and 
radio broadcasting. Unlike point-to-point communication, the sender (TV station) 
simultaneously broadcasts multiple messages (TV programs) to many receivers (TV sets) 
simultaneously via noisy channels. This implies that constructing a jointly optimal code 
with respect to the multiple channels may provide improved performance (i.e., higher 
capacity) than that of the time-sharing scheme, whereby separate optimally designed 
code are used for each channel. Actually, Cover showed that jointly optimized codes 
can have a larger capacity region, where error free communication becomes possible, 
than that of timesharing codes [2, 3]. However, his proof is non-constructive and the 
search for better practical codes for broadcast channels is still an important topic in 
information theory. 

The purpose of this paper is to devise and analyze an improved practical code 
for a broadcast channel by linearly combining Low-Density Parity-Check (LDPC) 
codes, which have been shown to provide nearly optimal performance for single 
channels [6, 10, 5]. For Reed-Solomon and BCH codes, which are standard suboptimal 
codes, it has been reported that combining codes linearly results in superior performance 
with respect to a timeshared transmission [18, 11]. This provides the motivation for the 
current study, investigating the performance of linearly combined LDPC codes. 

This paper is organized as follows. In the next section, we introduce the general 
framework for broadcast channels. Unlike simple communication channels, the optimal 
communication performance is still unknown for most broadcast channels, which would 
make it difficult to evaluate the performance of the proposed scheme. Therefore, we 
focus here on a simple case, termed a degraded channel, for which the capacity region 
has already been obtained. In section 3, an LDPC code based construction for degraded 
channels is introduced, and is subsequently analyzed in section 4 using methods of 
statistical physics. In section 5, the performance of the proposed scheme is evaluated 
by solving numerically equations that emerge from the analysis. The final section is 
devoted to a summary and conclusion. 
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Figure 1. (a): A single sender and two receivers broadcast channel, (b): The capacity 
region in the case of binary symmetric channels. The solid curve and the dotted line 
denote Cover's and timesharing limits, respectively. 

2. Broadcast channel 

In the general framework of broadcast channels, a single sender broadcasts a codeword 
composed of different messages to multiple receivers. For simplicity, we here restrict our 
attention to the case of a single sender and two receivers (Fig. 1), where one codeword 
X (N bits), comprising two messages Wi(RiN bits) and W 2 (R 2 N bits), is sent to two 
receivers. As each channel is noisy, receivers 1 and 2 obtain two corrupted codewords 
and 3^2, respectively; this is modeled by a conditional probability P(y±, 3^2 1<^)- The 
received corrupted codewords 3t and 3^2 are decoded by the respective receivers to 
retrieve only the message addressed to each of them. 

Analogously to the case of single channels, error free communication becomes 
possible if the corresponding code rate vector R 2 ) lies within a certain convex region, 
termed the capacity region, determined for a given broadcast channel -P(3\, 3^2^) 
using an infinite code length N [3, 1]. Evaluation of the capacity region is one of 
the fundamental problems in information theory; the problem is generally difficult and 
has not yet been solved in general except for a few special cases. 

A broadcast channel P(3^i, 3^2^) is termed degraded if there exists a distribution 
P'Wyi) such that 

= J2 pw*) • (i) 

The degraded channel is exceptional in the sense that its capacity region can be 
analytically obtained as the convex hull of the closure of all points (R\, R 2 ) that satisfy 

R 2 < i(u\y 2 ) (2) 

R 1 <I(X;y 1 \U) 1 1 

for a certain joint distribution P(U)P(X\U)P(y 1 ,y 2 \X); where the auxiliary random 
variable U has a cardinality bounded by \U\ < min{|A'|, |3^i|, |3^|}- This region is often 
called Cover's capacity [2, 3] region. Unfortunately, the derivation of Cover's capacity is 
non-constructive and offers little clue to design efficient practical codes. Thus, practical 
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codes for the degraded broadcast channel has been actively investigated in the network 
information theory [4]. 

In the case of binary symmetric channels characterized by flip probabilities p\ and 
p 2 , condition (1) reduces to an inequality p 2 > P\- Then, the expression of Cover's 
capacity is simplified to 



where a parameter < 5 < 1 specifies the optimal ratio between R\ and R 2 ; 5 * p = 
5(l—p) + (l — 5)p and H 2 (p) is Shannon's entropy H 2 (p) = —plog 2 p—(l—p) log 2 (l— p)- 

The solid convex curve in Fig. 1(b) shows Cover's limit, i.e., the boundary of Cover's 
capacity for the binary symmetric channels. The straight broken line corresponds to 
the timesharing capacity, i.e., the achievable capacity by concatenating two independent 
codeswords optimized for each channel separately. This is realized by using JV(1 — a) and 
Na bits of codeword X for encoding messages Wi and W 2 , respectively. Here, < a < 1 
is the code length ratio between the two messages. This simple concatenation and the 
limit achievable by this scheme are often termed the timesharing and the timesharing 
limit, respectively. The difference between Cover's and the timesharing limits indicates 
the capacity gain obtained by optimizing a code for the complete broadcasting system 
in comparison with respect to optimizing each of the channels separately. 

We have to emphasize that achieving the timesharing limit in practice is never 
trivial as there is no known practical code that saturates Shannon's limit even for a 
single channel. Therefore, the design of improved practical codes for broadcasting, by 
combining existing codes, devised for single channels, is an important research topic in 
coding theory [11]. 

3. Linearly combined codes 

Linearly combined codes is a well-known strategy for designing high performance 
communication schemes for broadcast channels using multiple linear Error- Correcting 
Codes (ECC) [18, 11]. In this scheme, the first N(l — a) bits of a codeword are obtained 
by linearly mixing two messages Wi and W 2 while the other Na bits are generated only 
from W 2 by some linear transformation. In both coding and decoding, all operations 
are typically carried out in modulo 2. This method has been developed for algebraic 
codes, such as Reed-Solomon and BCH, which are standard codes designed for relatively 
short code lengths. For these codes, it is reported that the minimum distance between 
codewords is larger than that achieved in the timesharing scheme, which implies higher 
robustness against channel noise [18, 11]. 

However, it is unclear whether a similar construction also offers better performance 
when different code types are used. Furthermore, it is theoretically interesting and 
important to examine whether a linearly combined code can saturates Cover's limit for 
infinite code length (N) or not. 



( 



R 2 < 1 - H 2 (5*p 2 ) 
R 1 <H 2 (5*p 1 )-H(p 1 ) 



(3) 
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Motivated by these questions, we investigate here the ability and limitations of 
linearly combined LDPC codes in the limit iV — > oo. 

An LDPC code is characterized by a parity check matrix. To devise a linearly 
combined coding scheme for LDPC codes, we define a parity check matrix in an upper 
triangular form 

-Co y 

where the sizes of the sub-matrices A 1 ,A 2 ,A 3 are [(1 — a)N — R±N] x (1 — a)N, 
[(1 — a)N — R±N] x aN, [aN — R 2 N] x aN, respectively. Further, we assume that 
Ai,A 2 ,A 3 have Kx,K 2 ,K 3 and C\,C 2 ,C 3 non-zero elements per row and column, 
respectively. Based on the parity check matrix, the generator matrix G T is constructed 

as 

< 6 > 

where Gf (i = 1,3) are constructed systematically to satisfy the constraints AiGj = 
(mod2) and G 2 is defined as — Aj [AiAj ] _1 [A 2 G 3 ]. The sizes of these matrices are 
(1 - a)N x RiN, aN x R 2 N and aN x R 2 N, respectively. 

The sender produces a codeword X by taking a product of the generator matrix G T 
and the original messages (Wi,yV 2 ) T . Receiving a possibly corrupted codeword, each 
receiver evaluates the syndrome vectors J, = Ay^ (i — 1, 2), which yield the parity-check 
equation Jj = A£ { . The message vector ^ can be thought of as having two separate 
segments denoted by u (up) and d (down) later on. The parameter a controls the 
error correction ability for the second message; the transmitted information redundancy 
increases with a (a > R 2 j (Ri + R 2 ))- The decoding problem for each receiver is to find 
the most probable messages, Sj and cr^, such that the parity check equation 



A 1 A 2 








(< = 1,2) (6) 



is obeyed, and using prior knowledge about the two noise vectors characterized by the 
two different channels. 

The second receiver has to estimate only the lower part of noise vector £ , which 
can be carried out using only the lower part of Eq.(6). However, we assume here that 
both receivers independently solve Eq.(6) using prior knowledge on their own channels 
since one can show that solving the whole equation provides the optimal estimation 
performance for both receivers. As Eq.(6) has the same form for receivers 1 and 2, we 
hereafter omit the subscript i = 1,2. 

For bit-wise minimization of the error probability the optimal estimation is given 
by maximizing the posterior marginal (MPM) 

& =arg max P(si | J), f ■ = argmaxP(a i | J). (7) 

Sie{o/i} ^e{o/i} 

An exact evaluation of Eq.(7) is generally hard; therefore, the belief propagation (BP) 
approximation scheme is widely used as a practical decoding algorithm. The latter has 
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been shown to be identical to the Thouless-Anderson-Palmer (TAP) approach in the 
current case [9]. 

4. Statistical mechanics 

4-1. Macroscopic analysis - performance evaluation 

In order to evaluate the typical error-correction ability of these codes in the limit 
N — > oo, we investigate the behavior of the MPM decoder using the established 
methods of statistical mechanics. We first map the current system to an Ising spin 
model with finite connectivity, by employing the binary representation {+1, —1, x} for 
the alphabet and operator instead of the Boolean one {0, 1, +}. This implies that the 
posterior probability P(s,cr\J) can be expressed as a Boltzmann distribution at the 
inverse temperature (3 — 1 using a Hamiltonian 

n(s,<r\J) = lim 1 7 J2 ^T(K-l ),J(K 2 )^(~ Jl(.Ki ),J(K 2 )'i II «i II 

{ {X{K{],J{K 2 )} i€L{K{) jej(K 2 ) 

)(l-a)N a N 
-F E Si-F^aj, (8) 
i=i j=i 

where 1{K) = (ii,i 2 ■ ■ • ,ix) denotes the combination of the K subscripts chosen from 
the i = 1, 2, •••,(]. — a) N possibilities without duplication (the order is ignored), and 
J{K) = j 2 ■ ■ ■ ,]k) is the K combination from j = 1, 2, • • • , aN chosen similarly. 

1 2 

The tensor Dj'iK^ j(k 2 ) becomes 1 when its subscripts agree with the positions of non- 
zero elements in the parity-check matrices Ai and A 2 , and otherwise. The tensor 
Dj( Ks ) similarly corresponds to A 3 . The first and second terms in Hamiltonian (8) 
correspond to Eq.(6) while the third and fourth terms are provided by the prior 
distribution of the noise. The field F represents the channel noise level; it is set to 
| ln(l — pi)/pi and | ln(l — p 2 )/p2 for the first and the second receivers, respectively. 

In order to simplify the calculation, we first employ the gauge transformation 
Si — >■ s^J 1 , Oj — > °~jifi J^. — > 1 an d J d . — > 1, which reduces complicated couplings 
expressed in the first and second terms in Hamiltonian (8) to simple ferromagnetic 
interactions. 

As the parity check matrices and noise vectors are generated randomly, we have to 
perform averages over these variables for extracting typical properties of the code. This 
can be carried out by the replica method 

— = (lnZ) A gi£d = lim„_+o(l/W) ln(Z n — l)^* u i d , where Z is the partition 
function and (• • represents an average over the parity check matrix A and the 

noise vectors £ u and £ d (i.e., the quenched variables). This gives rise to three sets of 
order parameters 

1 (l-a)N 

nt \ ^ V e ai Q am 

H{a\,a2,-,a m } j.j ■ ■ ■ °\ ; 

iV i=l 
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r { ai ,a2,—,a m } 



t{ai,a2,---,a m } 



-i aN 



N U 



i 



aN 



yz 



(7 ai (7 a ' 



(9) 



where a±, a 2 , ■ ■ • , a m denote the replica indices running from 1 to n, and their conjugates 
q{ ai ,a 2 ,-,a m }, ^{oi.oa.-.a™}, ^{oi.oa.-.a™}- The variables Z,- are introduced to express the 
constraint of the parity-check matrix A3 as 




n 3 



c. 



^ z E J( K3)/,^(K 3 )-(^+l)_ 



2tt 



(10) 



The variables and Y} are similarly introduced for A 2 and A 3 . 

In order to proceed further, one has to make an assumption about the symmetry of 
replica indices. Here we employ the simplest replica symmetric (RS) ansatz, expressed in 
the current case by q{ ai ,...,a m } = Qo I dx ir(x)x m , r {ai ,..., am} = r J dy p(y)y m , t {au ..., am} = 
to J dz (p(z)z m , where go, r o and r are the normalization constants to make n(x), 
p(y) and <f>(z) proper probability distributions over the interval [—1,1], respectively. 
Unspecified integrals are performed over [—1, 1]. We also assume a similar ansatz for 
the conjugate variables. A further complicated assumption about the order parameter 
symmetry is generally required in most disordered systems [12, 13]. However, the validity 
of the RS ansatz in the current system is strongly supported by a recent report on 
the absence of the replica symmetry breaking in gauged systems where Nishimori's 
temperature is used. The latter corresponds to using the correct priors in decoding [14], 
as performed in the current analysis. 

Under these assumptions, one obtains the free-energy 

T= (1-Ri-R 2 )\n2 



a 



K 2 

#i)(ln|l + n^LU' 

1=1 l'=l 



K, 



[a 



R 2 )i\n 1 + n^ 



1=1 



4> K 3 



+ (1 - a)Ci (In (1 + xx))^ + aC 2 (In (1 + yy)) pJ) + aC 3 (In (1 + zz))^ 
Tr s e^ F n(l + ^ 



+ (1 



a 



In 



1=1 



+ a ( In 



c 2 



c 3 



Tr a e^ F l[(l + am) Hil + az,) 



1=1 



v=i 



(11) 



where (• • -) p k denotes an integral of the form / IlfcLi dxkP(xk)(- ■ •) and (/(£))$ 
(l-p)/(+l)+p/(-l). 

Varying Eq.(ll), one obtains a set of saddle-point equations, 

6(x- tanh tanh^ 1 x t + £ U F 



7T{X) 

p{y) 
4>(z) 



5 (y — tanh 
5 (z — tanh 



E£i _1 tanh" 1 Vl + YSU tanh" 1 z v + £ d F 
ESi tanh_1 Vl + Efe 1 tanh 



Zl> 



£,p C, 2- 1 ,4> C 3 
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7T [X) 



K 2 



(12) 



p(y) = (s{y-Ul\x l uifr 1 1 yi')) nKlpK2 _ 1 

The overlaps Mu = n~Tjjv ^* ^£* U an< ^ = ^'^j serve as performance 

measures for the error-correcting ability. After solving the saddle-point equations (12), 
these can be calculated as 



M u = Jdh h^ & (h) sign(/i), M d = J dh h^ s (h) sign(/i), 
where distributions of effective fields h e ^(h) are evaluated as 



(13) 



ti£ s (h) = (S[h- tanh 
/$,•(//) = /8\h - tanh 



Ci 



5] tanh -1 x z + £F 



z=i 



C*2 C3 

2 tanh" 1 + 2 tanh" 1 5, + £F 
1=1 i'=i 



(14) 



4-2. Microscopic analysis - practical decoding 



As already mentioned, it is computationally hard to perform MPM decoding (7) exactly. 
Instead, the belief propagation (BP) algorithm [15] is widely used for a practical 
decoding in LDPC codes. Belief propagation has recently been shown to be equivalent 
to the Thouless- Anderson-Palmer (TAP) approach in spin glass theory [17, 9]. We also 
make use of follow this decoding algorithm in the current framework. 

The BP/TAP approach offers an iterative algorithm to approximately evaluate 
marginal posterior distributions based on local dependencies between syndrome and 
variables. These local dependencies can be uniquely identified with conditional 
probabilities. In the current system, these become: = P{ni = n|{J\J M }) and 
eft oc P(J M |n; = n, { «7\ J M }) where n\ and J M represent components of spin variables s, 
a and syndrome J, respectively; { J\J M } denotes the set of syndrome bits excluding \x- 
th component. As most syndrome and spin variables are not directly related, we assign 
the conditional probabilities only to pairs fil that have non-zero elements in the parity 
check matrix A. 

Evaluating the two types of conditional probabilities using directly connected 
components, the BP/TAP algorithm can be generally expressed as 

D Fn tt ~n 

llveM(l)\n Quh 



%i = 
%l — 



a 



111 L,n jeC (n)\l 



UjeC(fi) Uj) Tlj£C(ti)\l Qfij: 



(15) 



where M(l) and £(//) denote the sets of syndrome and spin variable indices that are 
directly linked to spin and syndrome indices / and //, respectively; represents 
the set of indices v e M(l) excluding \x and similarly for C(fi)\l and other sets. 
Normalization constants, and a^, are introduced to make and q™ t probability 
distributions of spin variable n. A field F is introduced to represent the prior probability. 
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Since spin variable n takes only two values ±1, it is convenient to express the 
BP/TAP algorithm using spin averages J2 n =±i n< i^,i and E n =±i n <?^z rather than the 
distributions and themselves. As the parity check matrix A is structured, it may 
be useful to assign different notation to the spin averages according to the submatrix 
to which the pair of indices \il belongs to. We use x^y^x and z^ to denote H n =±i n Q^i 
when the pair of indices fil belongs to A 1 , A 2 and A 3 , respectively. Similar notations 
XnhVui and z pi are used for J2 n =±i n Qui- Then, the BP/TAP algorithm (15), which is 
expressed as a set of functional equations, is reduced to a couple of nonlinear equations 



' x^i 


^tanh^col^^tanh^^ + F], 






< V/d 


= tanh[E^ A col (0V tanh- 1 y vl + E^col^ 


(16) 


< 


= tanh[E^ A eol (0 tanh_1 V* + ^ eA col (0V tanh " 








= sign( J M ) n i6j4 row (M)V x^ n jGA row (M) y^, 








= sign( J M ) Il ieA row (M) x ifl U jeA row M ^ 




(17) 


{ Zpl 


= sign( J M ) n j£A row (M)V z jlt , 







where A row (fi) and A co[ (l) denote the sets of non-zero elements in the fi-th. row and 
i-th column of matrix A, respectively. 

Eqs.(16) and (17) can be solved iteratively from appropriate initial conditions (prior 
means are usually chosen as initial states). Less then 50 iterations are typically sufficient 
for convergence. After obtaining the solutions, approximated posterior means can be 
calculated 

(sj) = tanh[ ^2 tanh -1 x vi + F] 

ueA^°\i) 

((7j) — tanh[ ^2 tanh -1 + ^ tanh -1 z vi + F], (18) 

ueA^U) yeA^{j) 

which provides the MPM estimators Sj = sign((sj)) and aj = sign((<7j)). 

It can be shown that the BP/TAP framework provides an exact result when 
the global structure of the connectivities is graphically expressed by a tree [15]. 
Unfortunately, it is still unclear how good are the approximations obtained when a 
given system does not admit a tree architecture. 

The graphical architecture of LDPC codes generally has many loops, which implies 
the BP/TAP framework does not necessarily offer a good approximation. However, it is 
conjectured, and partially confirmed, that a nearly exact result can be obtained, as long 
as no other locally stable solutions exists, when the parity check matrix A is randomly 
constructed and in the limit N — > oo; this is due to the fact that the typical loop length 
scales as O(lnA) for randomly constructed matrices, which implies that LDPC codes 
can be locally treated as trees ignoring the effect of loops [16]. 
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5. Results 

In order to theoretically examine the typical performance that can be obtained by the 
linearly combined coding scheme, we solved the saddle point equations (12). Since 
solving the equations analytically is generally difficult, we mainly resorted to numerical 
methods. The solutions were obtained by iterating the saddle point equations (12), and 
approximating the distributions by O(10 4 ) sample vectors. Less then 50 iterations were 
typically sufficient for obtaining a solution. 

Solving the equations for several parameter sets, assuming a > R 2 /(Ri + R 2 ), we 
found that the solutions can be classified into three categories depending on whether 
overlaps M u and are 1 or not. The first one is referred to the ferromagnetic (F) 
solution (M u = M(j = 1) corresponding to perfect retrieval for both messages Wi 
and W 2 . The half-ferromagnetic (HF) solution which is characterized by M u 7^ 1 and 
M d = 1 implies that only the second message W 2 is perfectly retrieved, while Wi is 
not. The last category, termed paramagnetic (P) solution, describes a decoding failure 
for both messages being characterized by M u 7^ 1, 7^ 1. The ferromagnetic solution 
always exits and is locally stable for C\ > 3 and C3 > 3, while one can find other 
solutions only for relatively higher noise levels. As the noise level increases, HF and P 
solutions emerge in this order. 

The solution that has the lowest free energy among the three becomes 
thermodynamically dominant. As the noise level p becomes higher (or the field F 
becomes weaker), the dominant state changes from F to HF and P in this order. Since 
receivers are required to retrieve only their own messages, the transition point between 
HF and P corresponds to the maximum noise level for error free communication in the 
second channel while maximum noise level for the first channel is given by the transition 
point between F and HF. 

However, this does not imply a successful decoding up to the critical points in 
practical time scales. Practical perfect decoding by the BP/TAP algorithm is possible 
only when no suboptimal solutions exist, which means that the practically achievable 
limit is given by the spinodal points of the HF and P solutions for the first and the 
second channels respectively; i.e., the point where new suboptimal solutions emerge. A 
similar phenomena has been reported before for similar systems [7, 8]. 

Fig. 2 shows the maximum noise levels for perfect decoding of the linearly combined 
coding method obtained for C2 = 4 and fixing C\ = C3 = 3; C2 = corresponds to 
the sharing scheme for which A 2 = 0. One can find that both optimal and practical 
performances of the MPM decoder are improved by the introduction of the additional 
submatrix A 2 , as anticipated, in spite of the fact that the parameter C 2 (= 4) is not 
optimally tuned. This result may induce the hope that Cover's limit can be saturated 
by optimally tuning the submatrices. 

However, our analysis contradicts this conjecture. Solving Eq.(12) in the limit 
C 3 — > 00 and C\ or C 2 — > 00 is feasible; it is known that the MPM decoder provides 
the optimal performance in this limit while practical BP/TAP decoding becomes 
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Figure 2. Optimal and practical performance of the MPM decoder calculated by 
methods of statistical mechanics for different a values. For the first channel, the 
optimal performance is given by the thermodynamical transition between F and 
HF solutions while the transition between HF and P solutions marks the optimal 
performance for the second channel. On the other hand, the practical performance 
is given by the spinodal points of the HF and P solutions for the first and the 
second channels, respectively. Monte Carlo solutions based on 10 4 sample vectors 
were employed for solving the saddle-point equation (12). The standard deviation 
values resulting from 10 trials are smaller than the symbol size. 

The black squares and the black circles denote the optimal and the practical 
performances for the linearly combined coding scheme, where code parameters are set 
to C\ = C3 = 3, C*2 = 4, R\ = R 2 = 1/4. Diamond symbols denote the maximum noise 
levels for decoding success by the BP/TAP algorithm, determined from 50 experiments. 
The error bars are smaller than the symbols. 

Broken lines denote the optimal and practical performances of the timesharing 
for corresponding LDPC codes. The two lines in the upper right are Cover's and 
timesharing capacities calculated in the information theory. 



difficult. The three solutions correspond to those already mentioned before, but can 
be analytically expressed as: 

• F solution: Both messages are decodable(M u = = 1). The corresponding 
solutions and free energy are 

tt(x) = 5(x — 1) 

p(y) = S(y - 1) (19) 
{ 0(5) = 5(5-1), 

T = -(1 - 2p)F. 

HF solution: Message W2 is only decodable(M u 7^ 1, = 1). 

(5(x-tanh^F)) ? 



7r(x) 


= 5(x- 


1) 


« p(y) 


= 5(y- 


1) 


4(z) 


= 5(z- 


1) 



7T{X) 

p(y) = 8(y- 1) 

<P(z) = 5(z-l). 



7i{x) = 8(x).p(y) = 5{y) 
i(z)=S(z-l), 



(20) 
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T = (1 - a- i?i)ln2 - (1 -2p)F+ (l-a)[plnp + (1 -p) ln(l-p)]. 



• P solution: Both messages are not decodable(M u ^ 1, ^ 1). 



' (f)( z ) = (5(z - tanh£F)) 5 f 0(z) = <?(£) 

< ,p(y) = (%-tanh6F)> f m=sm 

ir(x) = (5(x — tanh£F)) 5 , [ 7r(£) = S(x), 



(21) 



JF = (1 - R Y - R 2 ) In 2 - (1 - 2p)F +p\np + (1 - p) ln(l - p). 



Examining the critical condition for decoding success in each channel, and 
comparing the free energy of the solutions, one obtains the capacity region of the linearly 
combined coding scheme 



This is, unfortunately, identical to the timesharing capacity which can be achieved by a 
simple concatenation of two independent codes. This result implies that the advantage 
of the linearly combined coding scheme vanishes as the submatrices become dense and 
this method cannot saturate Cover's limit. 

6. Summary and Conclusion 

In this paper, we have examined the performance of linearly combined LDPC codes, for 
information transmission in a broadcast channel. Our analysis shows that the capacity 
of the suggested coding scheme is upperbounded by the timesharing capacity, in spite 
of the apparent improvement in both optimal and practical performance with respect 
to LDPC based timesharing codes characterized by finite connectivity values. 

The reason for the failure of linearly combined LDPC codes to saturate Cover's 
limit may be explained by the codeword structure produced by this scheme. In his proof, 
Cover optimized the code performance by introducing a specific structure termed the 
cloud coding, employing an auxiliary random variable U as in Eq.(2). In cloud coding, 
a codeword X is randomly generated according to P(X\U) around a cloud center IA 
sampled from P(U). Knowing this structure, one can use the cloud center U and the 
coset X c = X — U for encoding W2 and Wi, respectively. 

In the case of binary symmetric channels, the optimal cloud center U can be 
obtained by sampling iV bit unbiased vectors for which the entropy per bit can be 
maximized to 1. On the other hand, one can produce the optimal coset X c by 
independently and randomly generating each bit using a uniform bias < 5 < 1, 
which provides an entropy H 2 (5) per bit. 

In an ideal situation, a noise vector £ x which is biased with a flip probability p 1 is 
added to the coset X c in the first channel. This implies that the entropy of the received 
coset becomes H2(S * pi) per bit while the entropy of the noise vector is i?2(pi) per 
bit. Since one can use the difference between the entropies to convey the information 
of Wi, the capacity of the first channel becomes R± < H 2 {5 * p±) — H 2 (pi), which is the 



R 2 < a[l - H(p 2 )\ 
R 1 <(l-a)[l-H( Pl )}. 



(22) 
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second inequality of Eq.(2). On the other hand, for the second channel, characterized 
by a flip rate p 2 , the coset X c together with a channel noise £ 2 serves as a single noise 
vector for which the entropy becomes H 2 {b *P2) per bit. As the entropy of the received 
cloud center can be maximized to 1 per bit, this means that the capacity of the second 
channel is given by R 2 < 1 — H 2 (8 *p 2 ), which is the first inequality of Eq.(2). 

In linearly combined coding scheme ( g t)W2 + ( q 1 )Wi, ( g |)VV 2 becomes almost 

random, which may serve as the optimal cloud center. However, the second part )Wi, 
that corresponds to the coset, is somewhat structured, differing from the optimal choice 
of uniformly biased random vectors. 

In order to compare the structured coset with the optimal one, let us fix the 
maximum entropy per bit of ( ^ )Wi, which equals 1 — a, to that of the optimal coset 
H 2 (8). Then, one can show that the entropy of the corrupted coset with flip probability 
p per bit always increases from H 2 (p*5) to (1 — a) + aH 2 (p) = H 2 (5)+H 2 (p) > H 2 (p*5). 
This means that the critical rate of the first channel increases from H 2 (5 *p±) — H 2 (pi) 
to (1 — a) [1 — H 2 (pi)] while that of the second channel reduces from 1 — H 2 (p 2 ) to 
a [1 — H 2 (p 2 )]. This trade-off between the capacities of the two channels limits the 
performance of linearly combined coding scheme to the timesharing limit, that is always 
within Cover's capacity region. 

In conclusion, while the suggested linearly combined LDPC coding scheme provides 
an improved performance over LDPC based timesharing codes for finite connectivity 
constructions, in both theoretical and practical limits, it cannot go beyond the 
theoretical timesharing limit; for that to happen, different coding schemes should be 
examined. 
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